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I,  INTRODUCTION 


A,  BACKGROUND 

“We  must  challenge  every  assumption  and  search  for  new  and  better  ways  to  accom¬ 
plish  our  tasks.  We  must  refine  requirements,  conduct  innovative  operations,  and  op¬ 
timally  allocate  resources  to  achieve  efficiencies  and  recapitalize  the  Fleet.  ” 

CNO’s  2003  Leadership  Guidance 

The  cost  of  operating  Navy  ships  is  difficult  to  determine,  but  extremely  impor¬ 
tant  to  accurately  predict.  Under-funding  in  this  area  could  result  in  the  deferral  of 
equipment  replacement  and  spare  parts  replenishment/consumption,  ultimately  reducing 
the  Navy’s  current  level  of  readiness.  Over-funding  could  hinder  the  Navy’s  efforts  to 
recapitalize  assets  in  order  to  meet  future  threats.  As  the  quote  above  underscores,  the 
Navy  is  determined  to  more  accurately  predict  resource  needs  in  order  to  fully  fund  re¬ 
capitalization  efforts. 

Within  the  Operations  and  Maintenance,  Navy  (0&M,N)  and  Operations  and 
Maintenance,  Navy  Reserve  (0&M,NR)  appropriations  categories,  the  Mission  and 
Other  Ship  Operations  (IB IB)  sub-activity  group  provides  “resources  for  all  aspects  of 
ship  operations  required  to  continuously  deploy  combat  ready  warships  and  supporting 
forces  in  support  of  national  objectives”  (FY  2003  President’s  Budget).  The  IB  IB  sub¬ 
activity  group,  to  be  referred  to  as  Ship  Ops  throughout  this  paper,  has  its  resource  re¬ 
quirements  determined  by  the  OPNAV  N80  (programming)  staff.  The  IBIB  program 
area  is  divided  into  five  subprograms: 

Charter 

Fuel 

Utilities 

TAD  (Travel  and  Trainings  costs:  Temporary  Additional  Duty) 

OPTAR  (Operating  Target:  Includes  Repair  Parts  and  Consumables  purchases) 
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The  Ship  Ops  sub-activity  group  includes  the  costs  within  each  subprogram  for 
all  active  and  reserve  ships.  The  OPNAV  N82  office  responsible  for  this  sub-activity, 
also  known  as  the  Office  of  Budget  (FMB),  must  collect  inputs,  assess  requirements,  and 
provide  resources  as  necessary  to  support  the  requirements.  Figure  1  shows  the  percent¬ 
ages  of  the  total  FY  2004  Navy  Budget  for  O&M,  N  and  Ship  Ops. 


Figure  1:  Percentages  of  the  Total  Navy  Budget  for  0«&M,N  and  IBIB 


In  order  to  support  this  sub-activity,  N80  must  have  accurate  tools  to  forecast  re¬ 
quirement  costs  based  on  fleet  inputs.  The  OPNAV  staff  uses  the  Ship  Ops  model  to  de¬ 
termine  the  resource  requirements  for  ship  operations.  The  model  was  developed  by  the 
OPNAV  N80  (programming)  staff  several  years  ago  to  consolidate  inputs  from  numerous 
resource  sponsors.  By  consolidating  resource  sponsor  efforts,  the  model  advocates  a 
standardized  Navy  approach  to  determining  resource  requirements  for  Ship  Ops.  The 
existing  model  uses  three-year  moving  averages  and  average  number  of  ships  in  commis¬ 
sion  to  estimate  ship-operating  costs  for  the  upcoming  year. 
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B,  RESEARCH  DISCUSSION 

FMB  feels  the  eurrent  model  provides  a  good  first  estimate  of  costs,  but  wanted 
an  evaluation  of  the  model  as  a  predictor  of  actual  ship  operations  costs.  FMB  has  also 
expressed  an  interest  in  the  possible  development  of  a  more  accurate  (specifically  in  pre¬ 
dicting  SR  and  SO)  and  flexible  (to  include  operational  data  such  as  days  underway) 
model.  The  current  model  estimates  ship  costs  according  to  ship  class,  using  a  three- 
year  average  of  previous  years’  actual  operating  costs  per  ship  multiplied  by  the  average 
number  of  ship  years  per  class.  A  ship  year  is  defined  as  a  ship  in  commission  for  a  full 
year. 

Though  the  current  model  provides  FMB  with  a  good  first  approximation  of  oper¬ 
ating  costs  for  a  class  of  ships,  FMB  feels  the  model  could  be  improved  in  its  ability  to 
predict  SR  and  SO  cost.  Further,  the  current  model  does  not  provide  the  means  to  esti¬ 
mate  the  effect  of  increased  Operations  Tempo  (OPTEMPO)  in  the  middle  of  the  year. 
For  instance,  if  the  Abraham  Lincoln  Carrier  Battle  Group  (CVBG)  is  extended  on  de¬ 
ployment,  the  model  is  unable  to  predict  the  additional  costs.  While  FMB  can  produce 
some  numbers  to  estimate  additional  operating  costs,  these  numbers  are  not  very  defensi¬ 
ble  when  requesting  increased  funding. 

C.  OBJECTIVES 

The  intent  of  this  project  was  to  evaluate  the  current  model  used  by  the  Office  of 
Budget  (FMB)  to  forecast  future  operating  costs  for  Navy  ships  and  to  develop  an  im¬ 
proved  model  if  warranted.  Further,  we  sought  to  develop  relationships  between  opera¬ 
tional  data  and  costs  for  use  in  determining  supplemental  funding  requirements. 
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II.  USE  AND  LIMITATIONS  OF  THE  CURRENT  MODEL 


The  model  provides  FMB  with  a  summary  of  predietive  costs  to  be  used  for  re¬ 
source  requests.  The  model  has  been  in  use  for  about  five  years  and  there  has  not  been  a 
detailed  comparison  of  actual  costs  to  predicted  costs.  The  obvious  limitations  of  the 
model  are  scalability  and  flexibility.  The  summary  output  provided  by  this  model  can 
only  be  reduced,  at  the  lowest  level,  by  ship  class  and  sponsor.  The  user  cannot  easily 
input  proposed  operational  adjustments  to  multiple  ships  to  see  the  predictive  effects  on 
cost. 

Another  limitation  of  the  model  is  its  reliance  on  the  outputs  using  a  three-year 
moving  average  of  unit  costs.  This  method  provides  a  simple  means  for  making  cost  pre¬ 
dictions  and  rapidly  incorporates  the  effects  of  the  current  environment.  Drawbacks  to  its 
use  in  the  model  are  that  the  third  year’s  data  are  an  estimate  and  that  one  year  can  have  a 
significant  impact  in  the  unit’s  output  (e.g.,  while  planning  year  2003 ’s  costs,  the  pro¬ 
grammer  only  had  preliminary  cost  data  for  2002  based  on  the  past  6-9  months  from  the 
current  fiscal  year,  which  is  better  than  a  simple  prediction,  but  still  not  actual  cost). 

Before  analyzing  the  effectiveness  of  the  current  model  by  comparing  actual  with 
predicted  operating  costs,  the  following  section  will  detail  the  data  that  were  used  to 
compare  actual  costs  with  those  that  were  predicted  by  the  current  Ship  Ops  model. 
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III.  DATA  COLLECTION 


A,  COST  DATA 

Cost  data  were  used  in  this  projeet  for  two  purposes:  first,  to  evaluate  the  eurrent 
Ship  Ops  model’s  predietive  eapabilities;  and  seeond,  to  build  a  modified  model  and 
compare  its  predictions  to  that  of  the  existing  model.  We  used  various  versions  of  the 
current  model,  which  were  provided  by  FMB,  to  gather  historical  cost  data  for  the  ap¬ 
praisal  part  of  our  work.  The  Atlantic  and  Pacific  Fleets,  and  multiple  Type  Commanders 
(TYCOMs)  provided  the  information  for  the  modified  model. 

However,  we  had  certain  qualitative  and  quantitative  reservations  regarding  the 
data.  The  qualitative  problem  surfaced  when  we  were  assessing  the  current  model’s  ac¬ 
curacy.  It  appeared  we  were  not  looking  at  the  “first  estimates”  provided  by  the  Ship  Ops 
model  (by  first  estimate  we  are  referring  to  the  predictions  that  were  produced  for  the 
purposes  of  the  initial  budgeting).  Some  of  the  inputs  (e.g.,  price  growth)  might  have 
been  updated  during  the  fiscal  year  in  order  to  get  more  accurate  results.  The  benefit 
from  doing  this  is  that  more  accurate  estimates  can  support  the  argument  for  additional 
funding  when  the  need  arises.  Though  our  analysis  uses  only  actual  data,  our  conclusion 
could  be  slightly  or  significantly  different  if  we  compared  the  “first  estimates”  to  the  ac¬ 
tual  cost  figures.  Our  methodology  chosen  for  the  analysis  -  separating  the  effects 
caused  by  the  model’s  discrepancies  and  effects  stemming  from  input  inaccuracy  -  en¬ 
sures  that  the  basic  evaluation  remains  the  same  regardless  of  whether  we  used  the  “first 
estimates”  or  not.  The  problem  resulting  from  using  the  updated  predictions  is  that  the 
difference  caused  by  the  unreliable  inputs  may  be  more  significant  than  we  indicated. 

Quantitative  problems  were  mainly  caused  by  the  problem  of  data  availability. 
We  faced  this  problem  during  the  process  of  building  the  modified  model.  Since  we  used 
various  sources,  the  historical  cost  data  were  not  always  available  for  the  same  years. 
The  Navy  Energy  Usage  Reporting  System  (NEURS)  data  (days  underway  while  under 
various  Operational  Controls  (OPCON))  provided  by  LANTEET  is  only  available 
through  EY  96.  NEURS  data  provided  by  PACEET  go  back  through  EY  92.  Cost  data, 
contained  in  the  models  provided  by  EMB,  are  only  available  back  through  EY  94.  This 
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means  that  we  had  to  find  the  lowest  eommon  denominator,  that  is,  incorporating  only 
those  fiscal  years  into  the  project  where  “all”  the  data  were  accessible. 

When  conducting  our  initial  regression  analysis  it  became  evident  that  regressions 
that  did  not  include  price  growth  factors  were  more  significant  than  those  that  did  include 
them.  This  raised  suspicion  concerning  the  validity  of  the  inflation  factors  used  in  the 
model.  Further  investigation  by  FMB  concluded  that  in  order  to  obtain  a  weighted  aver¬ 
age  inflation  factor  to  be  used  in  the  model,  the  Inflation  Category  Codes,  which  are  as¬ 
signed  by  the  TYCOMs,  were  not  properly  assigned.  Therefore,  through  consultation 
with  FMB,  we  concluded  it  was  more  relevant  to  exclude  inflation  factors  in  regressions 
used  in  formulation  of  a  modified  model. 

B,  EMPLOYMENT  DATA 

In  order  to  determine  the  number  of  days  a  given  ship  (or  in  aggregate,  a  ship 
class)  was  underway  during  a  given  year,  we  obtained  data  from  the  NEURS  database. 
NEURS  is  a  program  the  Navy  uses  to  monitor  days  underway  for  all  surface  ships  (It 
primarily  records  the  amounts  of  fuel  used.  Eor  our  purposes,  days  underway  is  the  most 
relevant  information).  We  were  able  to  determine  if  a  ship  was  underway  while  on  de¬ 
ployment  or  underway  while  not  on  deployment.  With  these  data  we  are  better  able  to 
dissect  the  employment  of  ships.  When  performing  analysis  by  ship  class,  the  variables 
used  were  days  underway  while  deployed  (aggregation  of  all  deployed  OPCONs)  and 
days  underway  while  not  deployed.  Because  of  the  limited  data  points  available  for 
analysis  we  were  unable  to  use  the  additional  variable  (Deployed  to  Eifth  Elect)  without 
sacrificing  the  statistical  accuracy  of  the  regressions. 

C.  SHIP  CLASSES  CHOSEN 

Eor  our  analysis  of  the  current  model  we  chose  to  use  the  Pacific  Elect  DDG-5 1 

class,  because  of  the  amount  of  data  available.  It  is  a  large  class  and  it  represents  the 

growth  of  the  fleet.  In  Chapter  IV,  we  use  five  ship  classes  EEG-7,  DDG-5 1,  CG-47, 

DD-963,  and  EHA-1  to  give  an  overview  of  the  model’s  accuracy  at  the  ship  class  level 

for  the  period  EY97  through  EY02.  These  classes  provide  a  broad  representation  of  the 

surface  fleet.  DDG-5 1  represents  a  class  experiencing  growth  while  EEG-7  and  DD-963 
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are  classes  experiencing  contraction.  LHA-1  and  CG-47  are  ship  classes,  that  remain 
stable  in  numbers  throughout  the  period  analyzed. 

In  our  regression  analysis,  we  were  limited  in  the  ship  classes  we  were  able  to 
study.  For  example,  we  were  unable  to  obtain  submarine  employment  data  from  Naval 
Sea  Systems  Command  (NAVSEA  08).  NAVSEA  08  does  not  track  days  underway; 
they  maintain  information  similar  to  NEURS,  but  instead  of  days  underway  tracks  Effec¬ 
tive  Eull  Power  Hours  for  reimbursement  to  DOE.  We  performed  regression  analysis  on 
the  following  15  classes  of  ships  for  which  we  had  all  operations  data: 


AOE-1 

AOE-6 

MCM-1 

MHC-51 

EHA-1 

EHD-1 

LPD-4 

ESD-36 

ESD-41 

CG-47 

DDG-51 

DD-963 

PPG-7 

ARS-50 

CVN-68 

Table  1:  Ship  Classes  Used  in  Regression  Analysis 


Section  IV  analyzes  the  effectiveness  of  the  model  by  comparing  actual  with  pre¬ 
dicted  operating  costs.  Before  presenting  our  results,  this  section  details  our  methodol¬ 
ogy  and  analysis  application. 
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IV.  DATA  ANALYSIS 


A.  METHODOLGY 

In  this  section  we  will  discuss  the  methodology  for  evaluating  the  Ship  Ops 
model.  Generally,  the  model  creates  an  average  unit  cost  (per  ship  year  or  per  OP- 
MONTH),  and  then  uses  estimated  execution  data  to  generate  the  predicted  basic  re¬ 
quirement  for  the  next  year.  This  basic  requirement  is  then  adjusted  by  the  estimated 
price  growth  (percentage  growth  or  decrease)  and/or  by  the  estimated  incremental  cost  in 
order  to  derive  the  adjusted  requirement  for  the  given  year.' 

To  filter  the  inaccuracies  of  the  estimated  operational  and  financial  inputs,  we 
created  “predicted  from  all  actual  data”  (PFAD)  costs  for  ex-post  prediction.  Figure  2 
shows  the  structure  of  the  inputs  used  in  the  model  to  produce  these  quasi-predicted 
numbers.  The  PFAD  costs  demonstrate  what  would  have  happened  if  all  the  inputs  had 
been  absolutely  precise. 


*  Incremental  costs  are  one-time  costs  such  as  replacing  foam  mattresses  with  spring  mattresses.  Incre¬ 
mental  costs  can  be  determined  and  used  by  the  RS  or  CL  for  each  cost  element. 
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Figure  2:  Creation  of  “Prediction  from  all  Actual”  (PFAD)  Costs 
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DNUi// 

Actual  deployed  days 

Actual  not  deployed  months  *  30.4  =  I  3.760.1 8 


Actual  DIM 
I  1,134.001- 


Actual  DLW 
I  1,134.00K 


Actual  Burn  Rate 

I  533.66  K 


Calculated  DNUW 

-►I  2.626.1 8  K 


Actual  Burn  Rale 

I  15.00K 


LWBBL 

605,171.641' 


DUW  Fuel  Cost  (in  $1 ,000) 

? - H  524,401  - - ■ 


Actual  Fuel  Price 
I  $40.3^ 


JJWBBL 
39,392.64  h 


NDLIW  Fuel  Cost  fin  $1 ,000) 
? - H  51 ,588 1 - 


Actual  Fuel  Price 

I  540.3^ 


SFCost 

$54.8341 


Repair: 

Actual  OPMONTH  Calc.  OPMO  (12  months  ’  Ship  Years) -  OPAfOAITH 


I  174.301- 
Actual  Ship  Year 

I  1 6.00 1- 


Adi.  DUW 
1 07.62  h 


Adj.DUWtMO 

20%  of  30.4  days  =  I  6.08  K 

Burn  Rate 

avg.  of  nondepl.  DUW  a  DNUW  actual  burnrates  =  I  274.33 1- 


LfWBBL 

29,522.35$ 


DUW  Fuel  Cost  fin  $1 ,000y 
? - H  $1,190l^^ 


Actual  Fuel  Price 
I  $40.32K 


(Actual  SR  Cost  •  <ucr.Cost) / Ship  Yr 


Year-3  |  $1 8,274 1  -1,171| 


SR  Unit  Cost  Actual  Price  Growths 

[Y-31->[Y-21  [Y-21->[Y-1| 

I  $1 ,662 1  *  I  98.60%!  •  111.30% 


SR  Unit  Cost  in  Year-1  dollar 


$1,823.87! 


Year-2  !  $1 9,562 1  -1,285|  13.00 1 


$1 ,784.81! 


Year-1  !  $23 ,462 1  -831 1  14.00| 


$1 ,735.21 1 


Average:  I  $1 ,781 .30 1 


Actual  Ship  Yr 

!  16.00! - 

Avg.  SR  cost 

!$1,781  .30 1  - - 

Actual  Price  Growth  [Y-1  ]->[Y]  I 
!  99.99%!  J 


Base  Requirement 
' — I  $28,4981 — ■ 


Actual  Incr.  Cost 

I  $ol — ' 


SR  Cost 
i  $28.4981 
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Including  the  actual  cost  data,  we  have  three  numbers  for  comparison  for  each 
cost  element:  actual,  predicted,  and  PFAD.  The  model’s  total  inaccuracy  can  be  calcu¬ 
lated  by  subtracting  the  predicted  cost  from  the  actual: 


Model ’s  total  inaccuracy  =  Actual  cost  -  Predicted  cost 


In  this  way  we  get  the  difference  between  budgeted  (predicted)  and  incurred  (ac¬ 
tual)  costs,  which  is  our  focus.  However,  by  using  the  quasi-predicted  PFAD  costs,  we 
can  decompose  this  difference  into  its  two  main  components. 

First,  by  obtaining  the  difference  between  PFAD  and  the  predicted  costs,  we  de¬ 
termine  the  effect  of  data  inaccuracy  from  the  budgeting  process: 


Effect  of  source  data  inaccuracy  =  PFAD  -  Predicted  cost 


The  second  component  can  be  calculated  by  determining  the  disparity  of  the 
PFAD  and  the  actual  cost  figures.  This  difference  gives  us  important  information  about 
the  model’s  predictive  ability  without  the  noise  caused  by  imprecise  inputs. 


Effect  of  the  model ’s  method  =  Actual  cost  -  PFAD 


Summing  the  component  effects  determines  the  model’s  total  inaccuracy: 


Effect  of  source  data  inaccuracy  +  Effect  of  the  model’s  method  =  (PFAD  cost  -  Pre¬ 
dicted  cost)  +  (Actual  cost  -  PFAD  cost)  =  Actual  cost  -  Predicted  cost  = 

Model’s  total  inaccuracy 
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As  we  will  see  in  the  Results  seetion  of  this  ehapter,  these  two  component  effects 
sometimes  occur  in  the  same  direction  (i.e.,  both  underestimate  or  both  overestimate)  and 
combine  to  increase  the  total  difference.  Other  times  they  have  opposite  effects,  resulting 
in  a  smaller  total  difference  than  would  be  observed  by  summing  the  absolute  values  of 
the  component  effects. 

This  decomposition  method  sheds  light  on  problems  that  are  hidden  from  the  ob¬ 
server  who  only  takes  into  account  the  total  inaccuracy  of  the  model.  However,  due  to 
the  natural  variation  of  actual  costs,  improving  either  the  accuracy  of  the  source  data  or 
the  model’s  predictive  ability  will  not  guarantee  improvement  in  individual  years. 

B,  APPLICATION 

Our  project  focuses  on  improving  the  model’s  method  (reducing  the  second  com¬ 
ponent  effect),  but  we  will  discuss  some  input  precision  (first  component)  issues.  In  the 
second  part  of  the  Results  section,  we  use  hypothesis  testing  and  the  Mean  Absolute  Per¬ 
centage  Error  (MAPE)  to  examine  the  difference  between  the  PE  AD  and  actual  costs. 

1.  Hypothesis  Test 

Eor  our  analysis,  we  want  to  see  if  the  differences  between  actual  costs  and  model 
predictions  are  in  effect  random  deviation,  or  if  the  differences  are  statistically  significant 
and  a  pattern  exists  in  these  differences.  The  null  hypothesis  is:  the  mean  of  the  differ¬ 
ences  (Actual  -  PE  AD)  is  zero;  while  the  alternative  hypothesis  is  that  it  is  not  zero: 

Ho:  p  =  0 

Hp  p  0 

where  p  is  the  real  mean  that  we  do  not  know,  but  estimate  as  A .  We  selected  the  per¬ 
centage  error  as  the  basic  unit  for  the  test,  since  it  is  comparable  across  ship  classes  as 
well  as  years.  We  calculated  p-values  for  determining  the  probability  of  making  a  Type  I 
error  (rejecting  the  null  hypothesis  when  it  is  true).  The  p-value  is  derived  from  the  t- 
statistic,  calculated  the  following  way: 


^  Albright,  Winston,  Zappe;  Data  Analysis  &  Decision  Making  with  Microsoft  Excel;  Duxbury  Press,  2002; 
p.  441. 
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t  -  value  = 


X-^ 
st  4n 

where  s  is  the  sample  standard  deviation,  and  n  is  sample  size. 

The  p-value  is  then  determined  by  using  a  t-distribution  table  (degrees  of  freedom 
equals  n-\)  and  the  assumption  of  a  two-tailed  test,  sinee  we  are  interested  in  probable 
differenees  on  both  ends  of  the  distribution  (positive  or  negative).  From  the  obtained  p- 
value,  we  can  either  reject  the  null  hypothesis  (which  implies  that  the  mean  of  the  differ¬ 
ences  is  not  zero,  so  the  model  estimates  values  inaccurately)  or  accept  the  null  hypothe¬ 
sis  (which  produces  an  overall  good  prediction  or  insufficient  evidence  of  the  opposite). 

2.  Mean  Absolute  Percentage  Error  (MAPE) 

The  second  method,  the  MAPE,  is  more  frequently  used  for  evaluating  the  accu¬ 
racy  of  forecasting  models.  It  is  the  average  of  the  prediction’s  absolute  percentage  error. 
It  has  an  advantage  of  using  absolute  values  for  comparison,  which  eliminates  the  offset¬ 
ting  effect  of  opposing  (positive  and  negative)  component  differences.  The  MAPE  is  an 
absolute  value,  which  can  be  objectively  applied  for  comparing  the  relative  strength  of 
different  forecasting  models.  But  its  disadvantage  comes  from  the  fact  that  it  is  a  subjec¬ 
tive  measure  when  used  without  a  benchmark  for  comparison. 

C.  RESULTS 

To  demonstrate  our  evaluation  of  the  current  model  we  will  analyze  the  Pacific 
Elect  DDG-51  ship  class  for  EY  2002.  Excluding  CT  and  NSI  costs,  the  model  predicts 
the  total  0&M,N  cost  fairly  well  (see  Eigure  4).  The  model  overestimated  the  costs  by 
approximately  $17.7  million  ($113.1M  -  $95. 4M),  which  is  an  18.6%  inaccuracy  relative 
to  the  actual  cost.  The  component  effects  are  similar,  as  most  of  the  cost  elements  ex¬ 
ceeded  the  estimated  values. 
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SF 

SU 

SR 

SO 

Total 

Actual 

$47,841 

$12,553 

$23,849 

$11,147 

$95,390 

Predicted 

$58,175 

$11,612 

$27,410 

$15,931 

$113,128 

Predicted  from  All  Actual  Data 

$54,834 

$11,251 

$28,498 

$15,660 

$110,242 

Actual  -  Predicted 

-$10,334 

$941 

-$3,561 

-$4,784 

-$17,738 

-  Pred.  fr  Actual  -  Predicted 

-$3,342 

-$361 

$1,088 

-$271 

-$2,886 

-  Actual  -  Pred.  Fr  Actual 

-$6,993 

$1,302 

-$4,649 

-$4,513 

-$14,852 

->  model  total  inaccuracy 
->  source  data  inaccuracy 
->  model  inaccuracy 


Figure  4:  Cost  Summary  for  Pacific  Fleet  DDG-51  Ship  Class  FY  2002 


SF,  which  has  the  largest  weight  in  0&M,N  costs  (in  this  case  50.2%),  was  esti¬ 
mated  with  a  fair  result  (see  Figure  5,  21.6%  difference  between  the  predicted  and  aetual 
costs). 


SF  value _ Full  effect  Part  effect  Weight 

Actual  $47,841  *^^410,334  -21.60%  50.15% 

Predicted  $58,1 -$3,342  -6.98% 


Predicted  from  All  Actual  Data 

$54,834 

^-$6,993  -14.62% 

Predicted 

vs. 

Actuai 

difference% 

-  Predicted  w/ actual  DUWs 

$52,880 

-$5,039  -10.53% 

2,402 

vs. 

2,170 

-10.70% 

Predicted  wl  actual  Bm  Rates 

$59,035 

-$11,194  -23.40% 

1,239 

VS. 

1,254 

1.20% 

Predicted  w/ actual  Fuel  Prices 

$58,175 

-$10,334  -21.60% 

40.32 

vs. 

40.32 

0.00% 

Figure  5:  Prediction  Analysis  of  Pacific  Fleet  DDG-51  Ship  Class  Fuel 

Cost  for  FY  2002 


Applying  the  deeomposition  method  to  these  results  uncovers  some  of  the  reasons 
for  the  difference  between  actual  and  predicted  costs.  The  reason  for  inaccuracies  in  fuel 
(SF)  cost  estimates  is  not  as  straightforward  as  the  distinction  between  model  error  and 
source  data  error.  Since  analyzing  SF  cost  prediction  is  not  our  primary  focus,  we  will 
briefly  review  the  results. 

Figure  5  shows  the  effect  of  the  source  data  inaccuracy  that  at  6.98%,  seems  rea¬ 
sonable.  This  is  true  in  the  ease  of  burn  rates  and  fuel  priees,  but  less  eonvineing  in  the 
number  of  days  underway.  Fuel  priee  is  the  same  as  predicted,  sinee  ships  use  a  prede¬ 
termined  fixed  price  throughout  the  year  and  bum  rates  do  not  ehange  signifieantly  over 
time. 
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SU  value 


Full  effect  Part  effect 


Actual  $12,563 
Predicted  $11,612 
Predictedfrom  All  Actual  Data  $1 1,251- 
-  Predicted  wf  actual  price  growth  $1 1 ,664 
-  Predicted  w/ actual  OflWOtVTH  $11,200 


SR  value 


$889 

$1,353 


7.50% 

-2.87% 

10.37% 

7.08% 

10.77% 


Weight 

13.16% 


Actuai  difference'^ 


132.60%  vs. 

180.70  vs. 


133.20% 

174.30 


0.45% 

-3.67% 


Full  effect  Part  effect 


Actual 

$23,849'' 

-$3,581  -14.93% 

25.00% 

Predicted 

$27,410" 

$1,088  4.58% 

Predicted  from  All  Actual  Data 

$28,498- 

— ~ . -$4,649  -19.49% 

Predicted 

vs. 

Actual 

difference% 

Predicted  w/  actual  price  growth 

$27,870 

-$4,021  -16.86% 

98.40% 

vs. 

99.99% 

1.59% 

■  Predicted  w/  actual  shipym 

$28,972 

-$3,123  -13.09% 

16.25 

vs. 

16.00 

-1.56% 

■  Predicted  w/ actual  incr.  cost 

$28,483 

-$4,634  -19.43% 

-1,073 

vs. 

0 

ffDIViDI 

SO 

value 

Full  effect  Part  effect 

Weight 

Actual 

$11,147' 

-$4,784 

-42.92%  11.69% 

Predicted 

$15,931- 

.$271 

-2.43% 

Predicted  from  All  Actual  Data 

$15,660- 

- . "  ^  -$4,513 

-40.48% 

Predicted 

VS. 

Actual 

difference% 

■  Predicted  w/  actual  price  growth 

16,324 

-$5,177 

-46.44% 

98.70% 

vs. 

101.20% 

2.47% 

-  Predicted  w/ actual  shipyear 

15,692 

■$4,545 

-40.78% 

16.25 

vs. 

16.00 

-1.56% 

-  Predicted  w/ actual  mcr.  cost 

15,511 

■$4,364 

-39.15% 

419 

vs. 

0 

#DIV/0! 

Figure  6:  Prediction  Analysis  of  Pacific  Fleet  DDG-51  Ship  Class  SU, 

SR  and  SO  Costs  for  2002 

In  each  of  the  remaining  three  cases  (SU,  SR,  SO)  we  can  draw  similar  conclu¬ 
sions.  Though  the  proportion  of  inaccuracy  fluctuates  (7.5%,  14.9%,  and  42.9%  -see 
Figure  6),  all  show  that  original  prediction  problems  stem  mainly  from  the  model’s  calcu¬ 
lation  method  (effects  respectively:  10.4%,  19.5%,  and  40.5%).  Even  if  the  planner  had 
known  what  was  going  to  happen  in  the  coming  year  (in  terms  of  the  cost  drivers  and  ad¬ 
justments  ship  years,  operation  months  price  growths,  and  price  growths  and  incremental 
costs  respectively)  using  the  current  model’s  method,  he  would  have  arrived  at  almost  the 
same  result.  However,  in  an  individual  case  it  can  come  from  natural  variation  of  costs 
over  time;  using  across-the-board  examples  we  can  determine  whether  it  is  a  general  ten¬ 
dency  or  not. 

For  the  selected  ship  classes  and  for  each  of  the  years  from  1997  through  2002  we 
ran  comparisons  measuring  the  second  component  effect  (model’s  inaccuracy).  Percent¬ 
age  errors  and  the  calculated  measures  are  summarized  in  the  tables  below. 
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Year/ 

CL 

DDG- 

51CL 

CG- 

47CL 

DD- 

963CL 

FFG-7CL 

LHA- 

1CL 

1997 

-9.39% 

-6.49% 

-4.34% 

-8.87% 

18.24% 

1998 

-12.91% 

-0.73% 

6.75% 

-5.98% 

1 1 .30% 

1999 

1 .64% 

-2.37% 

-3.29% 

-0.31% 

13.93% 

2000 

15.36% 

22.97% 

29.19% 

19.41% 

16.71% 

2001 

-5.43% 

-4.84% 

0.06% 

-2.54% 

21.00% 

2002 

-15.57% 

-12.35% 

-17.20% 

-14.55% 

-3.51% 

Mean  =  1 .53% 

StDev=  12.82% 


t-value  =  0.65 


p-value  = 


0.5187 


MAPE  = 


10.24% 


Table  2:  Prediction  Appraisal  of  Selected  Ship  Classes’  Total  Costs 

Table  2  shows  the  overall  results  obtained  by  analyzing  the  seleeted  ship  elasses’ 
total  eosts  (excluding  CT  and  NSI).  The  calculated  p-value  (0.5187)  implies  strong  evi¬ 
dence  for  not  rejecting  the  null  hypothesis,  which  theoretically  means  insufficient  evi¬ 
dence  against  Ho,  but  practically,  it  yields  a  good  overall  result  that  implies  a  good  model 
on  the  total  cost  level.  However,  we  should  highlight  the  deficiencies  of  this  analysis. 
By  using  simple  averages  we  do  not  take  into  consideration  the  different  ship  classes. 

On  the  other  hand,  the  MAPE  shows  a  fairly  good  picture.  It  says,  across  our 
sample,  the  total  cost  was  predicted  with  an  average  error  of  10%.  As  mentioned  before, 
there  is  no  objective  method  to  evaluate  this  number.  It  is  just  our  perception  that  deter¬ 
mines  this  as  fairly  good. 

As  we  will  see,  the  hypothesis  test  determines  whether  or  not  the  model  makes 
mistakes  systematically  or  randomly.  On  the  other  hand,  MAPE  gives  details  about  its 
ex-post  precision,  regardless  of  the  possible  fact  that  the  model  was  inaccurate  more  fre¬ 
quently  in  one  direction  then  the  other. 

Using  the  same  methodology,  we  can  assess  the  precision  of  prediction  separately 
for  each  cost  group.  We  begin  with  the  fuel  cost,  see  Table  3. 
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Year/ 

CL 

DDG- 
51  CL 

CG- 

47CL 

DD- 

963CL 

FFG-7CL 

LHA-1CL 

1997 

3.24% 

-7.21% 

-3.84% 

-34.35% 

1 1 .64% 

1998 

0.94% 

5.17% 

-0.98% 

-30.03% 

25.12% 

1999 

-7.43% 

-10.19% 

-4.29% 

-1 .36% 

17.40% 

2000 

54.10% 

51 .04% 

41.89% 

51.72% 

36.12% 

2001 

-1 1 .39% 

-7.56% 

-9.72% 

-19.31% 

41.49% 

2002 

-14.62% 

-16.82% 

-1 1 .86% 

-26.00% 

15.69% 

Mean  =  4.62% 

StDev=  24.91% 


t-value  =  1 .02 


p-value  = 


0.3180 


MAPE  = 


19.08% 


Table  3:  Prediction  Analysis  of  Selected  Ship  Classes’  Fuel  Cost 

The  p-value  (0.32)  gives  quite  strong  evidenee  against  systemie  errors;  however, 
the  MAPE  shows  only  a  fair  result.  In  certain  ship  classes  (e.g.  FFG-7  or  FHA-1)  this 
inaccuracy  is  especially  significant  and  presents  systematic  patterns  (continuous  over- 
and  under-estimation  respectively).  Since  this  cost  group  has  the  most  obvious  connec¬ 
tion  to  OPTEMPO  (e.g.  days  underway)  it  is  important  to  note  that  actual  data  yield  the 
above  results.  We  feel  these  results  demonstrate  the  potential  for  improvement  in  the 
prediction  of  SF  cost. 
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Year/ 

CL 

DDG- 

51CL 

CG- 

47CL 

DD- 

963CL 

FFG-7CL 

LHA- 

1CL 

1997 

16.69% 

10.08% 

8.99% 

11.15% 

34.77% 

1998 

25.30% 

12.13% 

13.24% 

7.84% 

-44.10% 

1999 

7.11% 

-9.38% 

2.39% 

14.95% 

12.67% 

2000 

2.70% 

3.39% 

2.61% 

3.83% 

-4.03% 

2001 

9.36% 

-1 .86% 

2.54% 

-5.89% 

7.54% 

2002 

10.37% 

14.81% 

-1 1 .37% 

-0.79% 

4.52% 

Mean  =  5.39% 

StDev=  13.32% 


t-value  =  2.21 


p-value  = 


0.0348 


MAPE  = 


10.55% 


Table  4:  Prediction  Appraisal  of  Selected  Ship  Classes’  Utility  Cost 

Results  from  the  analysis  of  utility  eost  are  somewhat  surprising  (see  the  summary 
in  Table  4).  Although  the  MAPE  shows  the  best  results  among  all  eost  elements,  the  p- 
value  indieates  systematie  problems  with  the  model  at  96.5%  eertainty  level.  This  indi- 
eates  a  statistieally  significant  one-direction  deviation  from  the  actual  data,  which  is  eas¬ 
ily  observable  by  examining  a  graph  like  the  one  in  Figure  7. 
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As  the  example  above  also  shows,  SU  cost  is  mostly  underestimated  if  we  use  ac¬ 
tual  execution  data  as  inputs  to  the  model.  This  seems  to  be  permanent,  as  the  p-value 
confirmed,  but  whether  it  is  intended  or  not  we  don't  know.  An  intended  flaw  might  be 
explained  by  the  commonly  used  under-financing  technique  (i.e.,  90%)  in  the  beginning 
of  the  year  (when  the  model  is  mainly  used)  in  order  not  to  exceed  100%  of  the  obliga¬ 
tions  by  the  end  of  the  year,  so  as  to  avoid  overspending.  If  it  is  not  intended,  it  would  be 
worth  analyzing  more  closely.  In  our  view,  we  think  we  are  observing  one  of  the  disad¬ 
vantages  of  moving  average,  which  happens  if  there  is  a  continuous  upward  or  downward 
trend  in  the  data,  where  moving  average  under-  and  over-estimates  respec¬ 
tively.  Correcting  this  would  probably  not  require  big  changes  in  the  model  (just  add¬ 
ing  the  average  difference  to  the  prediction  in  the  case  of  underestimation),  but  it  will 
work  properly  while  the  (upward)  trend  continues,  otherwise  it  would  have  the  opposite 
effect  by  causing  further  inaccuracies  if  the  trend  reverses.  The  planners  probably  have 
more  information  about  future  trends  based  on  which  they  can  decide  whether  or  not  they 
are  better  off  with  the  correction. 
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Year/ 

CL 

DDG- 
51  CL 

CG- 

47CL 

DD- 

963CL 

FFG-7CL 

LHA- 

1CL 

1997 

-24.42% 

1 .59% 

1 .87% 

9.55% 

36.91% 

1998 

-55.64% 

-19.26% 

4.87% 

8.15% 

-15.24% 

1999 

-2.52% 

-4.39% 

2.82% 

7.80% 

-10.94% 

2000 

1 .59% 

18.78% 

25.65% 

3.45% 

20.70% 

2001 

0.60% 

-0.17% 

-1.12% 

-5.79% 

6.82% 

2002 

-19.49% 

-13.95% 

-30.57% 

-17.89% 

-51 .00% 

Mean  =  -4.04% 

StDev=  19.94% 


t-value  =  1.11 


p-value  = 


0.2763 


SR 


MAPE  = 


14.12% 


Year/ 

CL 

DDG- 
51  CL 

CG- 

47CL 

DD- 

963CL 

FFG-7CL 

LHA- 

1CL 

1997 

-44.46% 

-41.52% 

-35.06% 

-10.34% 

-18.13% 

1998 

-11.16% 

6.50% 

28.34% 

6.07% 

26.85% 

1999 

23.85% 

24.20% 

-17.26% 

-25.70% 

23.09% 

2000 

-57.40% 

-21.66% 

22.98% 

6.54% 

5.64% 

2001 

-6.17% 

-9.47% 

25.24% 

29.83% 

8.95% 

2002 

-40.48% 

-17.52% 

-16.03% 

6.82% 

-22.95% 

Mean  =  -5.01% 

StDev  =  24.82% 


t-value  =  1.11 


p-value  = 


0.2775 


SO 


MAPE  = 


21.34% 


Table  5:  Prediction  Analysis  of  Selected  Ship  Classes’  SR  and  SO  Cost 


We  will  discuss  the  last  two  cost  elements  together,  because  they  are  calculated 
using  the  same  method,  namely  based  on  ship  years  (number  of  ships  in  commissioned 
status  in  a  given  year).  As  shown  in  Table  5,  their  p-values  are  very  similar,  at  a  mini¬ 
mum  showing  a  lack  of  sufficient  evidence  against  systematic  errors.  Despite  the  fact  that 
this  statistical  test  that  shows  the  errors  are  evenly  distributed,  there  are  significant  inac- 


21 


curacies,  especially  in  the  prediction  of  SO.  These  fairly  high  (in  our  judgment)  MAPE 
results  underpin  the  need  for  some  improvement  in  these  eases. 

Comparison  of  results  aeross  eost  elements  is  debatable  due  to  the  different  char- 
aeteristies  of  spending.  But,  eomparing  MAPE  results  indieates  the  possibility  of  im¬ 
proving  eost  estimation  in  the  last  two  oases  by  inoorporating  some  kind  of  operational 
data  into  the  model. 

After  analyzing  the  ourrent  teohniques  to  determine  predioted  oosts  at  the  speoial 
interest  item  and  ship  olass  level,  the  next  chapter  is  our  attempt  to  improve  upon  the  our¬ 
rent  methods  of  prediotion. 
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V.  MODIFIED  MODEL  PROPOSAL 


A,  INTRODUCTION 

In  this  section  we  will  discuss  our  findings  for  developing  a  modified  Ship  Ops 
model.  As  previously  stated,  our  research  focuses  on  improving  the  predictive  capability 
of  the  current  model  in  the  Special  Interest  Items  of  SR  (Repair  Parts)  and  SO  (OPTAR, 
Other).  We  will  use  as  a  benchmark  for  comparison  the  Mean  Absolute  Percentage  Error 
(MAPE)  analysis  completed  in  Chapter  IV.  Our  modified  model  will  be  compared 
against  the  current  model  to  determine  whether  we  have  succeeded  in  improving  the 
model’s  predictive  capability. 

The  primary  flaw  with  the  current  model  is  that  there  are  no  cost  drivers  other 
than  Ship  Years.  In  essence  the  model  treats  all  costs  as  fixed,  based  on  a  ship  being  in 
commission  during  a  given  year.  Our  modified  model  seeks  to  identify  the  fixed  cost  (a 
constant  in  the  cost  equation)  and  cost  drivers  related  to  operations  that  could  reveal  the 
underlying  variable  cost  of  operating  ships.  In  order  to  do  this  we  have  collected  opera¬ 
tional  information  from  the  NEURS  database  which  identifies  a  ship’s  days  underway. 
Eurther  segregation  of  the  data  is  possible  when  one  considers  the  OPCON  information 
found  in  the  NEURS  database. 

In  the  event  we  could  not  determine  a  relationship  between  costs  and  operational 
variables,  we  looked  to  improve  on  the  current  model’s  MAPE  by  finding  relationships 
between  cost  data  and  fiscal  year  (EY).  In  most  ship  classes,  we  determined  a  statisti¬ 
cally  significant  relationship  exists  between  costs  and  the  EY.  This  is  especially  relevant 
given  the  uncertainty  surrounding  the  current  inflation  factors  (discussed  in  Chapter  III). 
By  using  EY  as  an  independent  variable,  we  are  able  to  incorporate  the  historically  real¬ 
ized  rate  of  inflation  without  inputting  an  arbitrary  inflation  factor. 

In  selecting  which  regressions  to  use  in  our  modified  model,  we  chose  the  equa¬ 
tion  that  resulted  in  the  lowest  MAPE.  In  some  cases,  we  were  unable  to  find  a  relation¬ 
ship  between  costs  and  operational  data.  In  other  cases,  we  found  marked  improvement 
by  including  operational  data  as  drivers  for  forecasting  costs.  Our  modified  model 
incorporates  these  improvements,  where  available,  with  the  current  method  of  using 
three-year  averages.  We  have  determined  that  for  SR,  our  modified  model  demonstrates 


averages.  We  have  determined  that  for  SR,  our  modified  model  demonstrates  its  im¬ 
provement  over  the  eurrent  model  through  its  lower  overall  MAPE  (13.39%  for  the  modi¬ 
fied  model  vs.  20.27%  for  the  eurrent  model)  as  well  as  a  MAPE  for  each  ship  class  that 
is  lower  or  equal  to  the  current  model.  Eor  SO,  we  were  unable  to  produce  significant 
improvement  in  MAPE  when  compared  with  the  original  model. 

B,  DEVELOPING  THE  MODIFIED  MODEL 

This  section  (Tables  7  and  8)  presents  the  regressions  that  were  found  to  have  the 
lowest  MAPE  for  each  of  the  ship  classes  analyzed.  Regressions  were  run  to  find  rela¬ 
tionships  between  repair  parts  (SR)  cost,  consumable  (SO)  costs  and  operating  data.  An 
independent  variable  for  the  year  was  considered.  Referred  to  as  “EY,”  this  variable 
aimed  to  include  trends  from  year  to  year,  to  include  inflation.  An  indicator  variable  was 
included  to  differentiate  between  Pacific  and  Atlantic  Elect  ships  when  regressions  were 
run  on  all  the  ships  of  a  class.  This  variable  was  referred  to  as  “Pac  Pit.”  This  variable 
has  a  value  of  either  “1”  for  a  Pacific  Elect  ship  or  “0”  for  an  Atlantic  Elect  ship.  This 
variable  was  not  included  when  the  regressions  were  done  for  the  individual  fleets  since  it 
was  not  required. 

Based  on  the  information  in  the  NUERS  database,  five  possible  independent  vari¬ 
ables  could  be  considered.  The  first  was  days  underway  while  not  deployed  and  was 
identified  as  “EIW  not  dep.”  There  were  three  variables  to  consider  for  days  underway 
while  deployed.  Days  underway  deployed  to  the  Pifth  Elect  Area  of  Responsibility 
(AOR)  are  identified  separately  in  the  NEIERS  database  by  OPCON  code  17.  The  vari¬ 
able  representing  this  is  “code  17”  in  the  following  regressions.  When  ships  were  de¬ 
ployed  but  not  to  the  Pifth  Elect  AOR,  these  days  were  represented  by  the  variable  “EIW 
dep  not  17.”  Pinally,  the  variable  “Total  UW  deployed”  is  the  summation  of  the  previous 
two  variables.  The  last  variable  “Total  UW”  considers  the  total  number  of  days  under¬ 
way  deployed  and  not  deployed. 

Some  exceptions  apply.  Due  to  the  lack  of  data  points,  regressions  by  class  do 
not  consider  whether  a  ship  is  deployed  to  the  Pifth  Elect  or  not,  only  that  it  is  underway 
deployed.  Purther,  in  order  to  keep  with  the  model’s  current  convention  of  computing 
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unit  cost  for  SR  and  SO  and  then  multiplying  by  the  number  of  Ship  Years,  we  have  de- 
eided  to  use  the  dependant  variable  SR  per  ship  (or  SO  per  ship)  when  determining  the 
equation  to  prediet  eosts  by  elass. 

To  summarize,  the  variables  used  in  the  following  regressions  and  their  meanings 
are  as  follows: 


25 


Dependent  Variables 

SR 

A  dependent  variable  to  estimate  repair  parts  eosts  for  a 
ship  in  the  class  when  using  “by  hull”  data. 

SO 

A  dependent  variable  to  estimate  SO  for  a  ship  in  the 
class  consumable  costs  for  a  ship  in  the  class  when  using 
“by  hull”  data. 

SR  per  ship 

A  dependent  variable  to  estimate  SR  costs  when  using 
class  data. 

SO  per  ship 

A  dependent  variable  to  estimate  SO  costs  when  using 
class  data. 

Independent  Variables 

FY 

An  independent  variable  representing  the  current  fiscal 
year.  Fiscal  Year  2000  was  used  as  the  base  (00).  There¬ 
fore  fiscal  year  1999  is  represented  by  a  negative  one  (-1) 
and  fiscal  year  2001  by  a  positive  one  (1). 

Pae  Fit 

A  binary  (one  or  zero)  indicator  variable  to  represent  the 
fleet  in  which  a  ship  is  home  ported.  A  ship  assigned  to 
the  Atlantic  Fleet  would  have  a  value  of  zero  and  one  as¬ 
signed  to  the  Pacific  Fleet  would  have  a  value  of  one. 

UW  not  dep 

Represents  the  days  spent  underway  and  while  not  in  a 
deployed  status.  In  the  NUERS  database  this  is  repre¬ 
sented  by  the  time  spent  in  code  eight. 

Code  17 

Represents  the  days  underway  on  deployment  while  in 
the  5**'  Fleet  AOR.  This  time  is  represented  by  code  17  in 
the  NUERS  database. 

UW  dep  not  17 

Represents  the  days  spent  underway  and  on  deployment 
when  operating  in  areas  SO  than  the  5*  fleet  AOR.  This 
is  represented  by  the  code  nine  in  the  NUERS  database. 

Total  UW  deployed 

Is  the  summation  of  the  days  under  “Code  17”  and  “Total 
UW  deployed.”  This  represents  the  total  number  of  days 
underway  while  in  a  deployed  status. 

Total  UW 

Represents  the  total  number  of  days  a  ship  was  underway 
in  a  year.  It  is  the  summation  of  the  time  spent  in  codes 
eight,  nine  and  seventeen  in  the  NUERS  database. 

Total  UW  /  SY 

The  total  days  underway  for  a  class  during  a  year  divided 
by  the  ship  years.  This  represents  the  average  number  of 
days  underway  per  ship. 

Table  6:  Variables  used  in  Regressions 
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Multiple  regressions  were  run  in  Minitab  (a  eommereial  statistical  software  pack¬ 
age)  to  consider  the  various  combinations  of  these  variables.  In  order  to  find  any  rela¬ 
tionships  that  exist  across  an  entire  class,  the  ships  were  aggregated  by  class  and  fleet. 
Then  the  ships  were  divided  into  their  respective  fleets  and  further  regressions  were  per¬ 
formed  to  find  any  relationships  that  were  fleet  specific. 

There  are  a  few  exceptions  to  this  practice.  Only  ships  from  the  Atlantic  Fleet 
were  considered  for  the  CVN-68  class.  Data  for  the  Pacific  Fleet  ships  of  this  class  were 
not  available.  The  MCM  class  does  not  have  ships  assigned  to  the  Pacific  Fleet.  Ships 
are  home  ported  in  the  Atlantic  Fleet,  Bahrain,  and  Japan.  Although  assigned  to  Japan, 
for  budgeting  purposes  these  ships  are  considered  part  of  the  Atlantic  Fleet.  Regressions 
performed  on  this  class  of  ship  were  separated  by  homeport:  Atlantic,  Bahrain,  and  Japan. 
The  MHC  class  had  a  similar  issue  since  these  ships  are  only  home  ported  in  Bahrain. 
Detailed  analysis  of  regressions  can  be  found  in  the  appendices  of  our  full  MBA  report. 


C.  EVALUATING  OUR  MODEL 

We  established  which  classes  of  ships  have  demonstrated  a  significant  relation¬ 
ship  to  either  an  operational  variable  (days  underway)  or  a  sequential  variable  (FY).  We 
constructed  our  modified  model  based  on  the  premise  that  if  we  lower  the  MAPE  for  any 
portion  of  the  model  we  improve  the  predictive  capability  of  the  model.  With  that  in 
mind.  Tables  7  and  8  demonstrate  which  classes  (in  which  fleets)  have  a  statistically  sig¬ 
nificant  relationship  with  a  variable  not  included  in  the  current  model  that  could  improve 
the  predictive  capability  over  the  current  model. 
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SO 

Best 

Value 

Best  Method 

Be;t  Method  Equation 

Atlantic  Fleet 

AOE-ICL 

10.10% 

OFipal  Model 

3-year  averape 

A0E-6CL 

15,10% 

Repression  by  HULL 

SO  =  383230  - 35086  FY+ 2433  Total  UW 

iC-51CL 

30.00% 

Repression  by  HULL  Combined 

SO  =  131350  +46602  FY 

LHA-ICL 

],10% 

Repression  by  Class 

SOpe[5hip=  245],304  +  118,0T14FY 

LHD-ICL 

O.ffl 

Repression  by  Class 

SOpe[3bip  =  2281,05]  +  125.4181  FY 

LPD4CL 

10.30% 

Repression  by  HULL 

SO  =  ]53]10  +43124  FY 

LSD-36CL 

21,50% 

Repression  by  HULL  Combined 

SO  =  863234  +  226643  FY 

LSD41CL 

20.50% 

Repression  by  HULL  Combined 

SO  =  3844T1  +46386  FY+3M1PacFlt+  1803  TotalUW 

CG4/CL 

6,40% 

Repression  by  Class 

SO  per  ship  =  868,]325  +  36,6^2  FY 

DDG-51CL 

O.Mo 

Repression  by  Class 

SOper3hip=T11.38T  +  18.T4133FY 

DD-963CL 

6.00% 

Repression  by  Class 

SOpership  =  T54,3822  +  18.24034  FY 

FFG-/CL 

3,Mo 

Repression  by  Class 

SO  per  ship  =  61T, 0314  +24,24533  FY 

ARS-50CL 

].00% 

Repression  by  Class 

SO  per 3hip=  463.818  +45.26488  FY 

CVN-68CL 

0,52% 

Onpinai  Model 

8-year  averape 

P  jfi'fic  Fleet 

A0£4C[ 

im 

OfifiinatMl 

3-year  averape 

A0£-eC[ 

mo% 

RmssionhfMCofnM 

SO=23dd2^+58564/PacFf+33l2MtitV 

turn 

im 

fiepresslofibyClass 

sopeisfiip'-mm + mm  FY+nm  loiam/SY 

m 

m 

Repression  by  Class  Combi'fied 

SOpe[sf}lp--Wm+imfYUm3Pacf! 

LPm 

m 

Repression  by  Class 

SOpeisPip--  f333./53+8f.f52F8FY 

tsmt 

im% 

Repression  by  Class 

SOpersbr  iim+85.mifr 

ma 

i9.m 

Repression  by  HLILL 

SO  =  5f3888+3846MtiW 

CG-m 

m 

Repression  by  RLILL  Combined 

SO=5f0000+F022fFY+2MPaGFf+mfIotalLIW 

mm 

mo% 

Repression  by  HLILL 

SO = im + 40860  FY+ 4890  titV  W.D, + 5099  mDeplM  f/+  3320  Code  iJ 

mm 

am 

Repression  by  Class  Combined 

SOpeisPip--  8/6.4264  + 42.3440/ FY 

ffG-ja 

io.m 

Repression  by  Class  Combined 

SOpersbip=/04.08/2+36.8e082Fy 

mML 

iw% 

Repression  by  Class  Combined 

SOpersbip=4/3.42/f+46.69583FY+23/.9l25PacFf 

cmia 

20.00% 

Oriplnal  Model 

3-year  averape 

Table  7:  Best  MAPE  by  Type  of  Regression  SO 
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SR 

Beet 

Value 

Beet  Method 

Best  Method  Equation 

Atlantic  Fleet 

A0E-1CL 

m% 

Oripal  Model 

3-year  average 

A0E-6CL 

mn 

RedFessionhv  Class 

SR  persbp  1662.023  +  92.30492  FY 

mcL 

13m 

OFlpal  Model 

3-year  average 

IHC-51CL 

40.00% 

RepFesslonbyHULLComied 

SR =  492140  +  164223  FY 

LHA-1CL 

15.20% 

Repfesslon  by  Class  ComPed 

SR  per  sbip=  2148.265  +  91. 33448  FY 

LHD-1CL 

0.63% 

OFpal  Model 

3-year  average 

LPD4CL 

lo.m 

OFpal  Model 

3-year  average 

LSD-36CL 

11.94% 

OFpal  Model 

3-year  average 

LSD41CL 

12.64% 

OFipal  Model 

3-year  average 

CG47CL 

9.90% 

OFpal  Model 

3-year  average 

DDG-51CL 

3.90% 

RedFesslon  by  Class 

SR  per  sbip  =  1328  -  98.0024  FY 

DD-963CL 

4.40% 

RedFesslon  by  Class 

SRpersbip  =  1958.262  +65.34286  FY 

FFG-7CL 

3.00% 

RedFesslon  by  Class 

SR  per  sbip  =  1450.922  +43.02232  FY 

ARS-50CL 

11.90% 

RedFesslon  by  HULL 

SR =  414091  +48212  FY 

CVM-eSCL 

26.90% 

RedFesslon  by  HULL 

3R  =  3332599  +  231389  FY+  23395  TotalUW 

Patific  Fleet 

A0E4C1 

nm 

fiedressffl  by  HiiLL  Combined 

SR=M2+2P6Fy-MPasFf 

nmcL 

U.JOl 

RefjfembfMComU 

SR = mn-  Pac  fl + mi  m  nol  deg + 5232  2olal  LIW  deployed 

m-KL 

nm 

Regression  by  Class 

SRpersbp  2349.502+ 226.3022 FY 

Lmt 

im% 

Original  Model 

3-year  average 

LPm 

am 

Original  Model 

3-year  average 

ismL 

m% 

Regression  by  IfliLL 

SR =  232205 + 220246  FY +  6200  LAV  nol  dep + 3526  Folal  Dep  LiyV 

ismi 

ilool 

Regression  by  IfliLL 

SR =  882305- 56488  FY 

CG-m 

m% 

Original  Model 

8-year  average 

mm 

nm 

Original  Model 

3-year  average 

mm 

Regression  by  Class 

SRpersbip=  2033.559 +  222. 2649  FY 

FFS-/C[ 

m% 

Regression  by  Class 

SRpersbip=  2328.088+53.5FY 

mm 

13.60% 

Regression  by  HULL  Combined 

SR=  424092  +52624  FY+252e22PasFf 

mm 

22.23% 

Original  Model 

3-year  average 

Table  8:  Best  MAPE  by  Type  of  Regression  SR 
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We  have  demonstrated  that  in  some  eases  the  current  model  is  the  most  accurate 
means  of  predicting  costs  (lower  MAPE  or  no  significant  regressions  were  found),  while 
in  other  cases  a  driver  other  than  ship  years  is  more  appropriate.  Tables  9  and  14  show 
the  actual  cost  by  class  and  fleet,  the  PFAD  (the  best  possible  output  of  the  current 
model),  and  the  modified  model’s  predicted  cost  for  2002,  2001  and  2000. 
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SR  ■  2002 

Actual  Cost 

Weighting 

PFAD 

PFAD 

Weighted 

MAPE 

Best 

Method's 

Prediction 

Best 

Method's 

Weighted 

MAPE 

Atlantic  Fleet 

A0E-ia 

13,343 

0.99% 

$3,092 

0.08% 

$3,092 

0.08% 

AOE-eCL 

$792 

0.23% 

$1 ,113 

0.07% 

$1 ,111 

0.07% 

MCM-ia 

$9,176 

2.72% 

$7,223 

0.74% 

$7,223 

0.74% 

MHC-5ia 

$1 ,316 

0.39% 

$2,761 

0.20% 

$1 ,641 

0.08% 

LHA-ia 

$6,846 

2.03% 

$4,248 

1.24% 

$4,662 

0.95% 

LHD-ia 

$9,015 

2.67% 

$9,327 

0.09% 

$9,327 

0.09% 

LPD-4CL 

$4,351 

1.29% 

$4,129 

0.07% 

$4,129 

0.07% 

LSD-36CL 

$876 

0.26% 

$790 

0.03% 

$790 

0.03% 

LSD-41  CL 

$6,714 

1.99% 

$5,032 

0.67% 

$5,032 

0.67% 

CG-47CL 

$40,254 

11.94% 

$36,397 

1.27% 

$36,397 

1.27% 

DDG-51CL 

$28,455 

8.44% 

$25,821 

0.86% 

$21 ,055 

2.97% 

DD-963CL 

$21 ,029 

6.24% 

$20,728 

0.09% 

$21 ,934 

0.26% 

FFG-7CL 

$22,554 

6.69% 

$21 ,235 

0.42% 

$23,057 

0.15% 

ARS-50CL 

$832 

0.25% 

$740 

0.03% 

$1 ,023 

0.05% 

CVN-68CL 

$32,033 

9.50% 

$24,269 

3.04% 

$26,715 

1.89% 

Pscific  F/eet 

AOE-1CL 

$2,m 

0.80% 

$2,107 

0.22% 

S3, -111 

0.11% 

AOE-eCL 

$2,414 

0.72% 

$2,083 

0.11% 

12,493 

0.02%> 

Lm-1CL 

$%22e 

1.55% 

$7,831 

0.52% 

S3, 106 

0.55% 

LHEA1CL 

$7,0&8 

2.10% 

$7,433 

0.12% 

$7,433 

0.12% 

LPEA4CL 

$^,178 

1.54% 

$5,852 

0.18% 

$5,852 

0.18% 

LSEA3eCL 

$1,348 

0.58% 

$1,630 

0.03% 

$2,488 

0.18% 

LSEA41CL 

$4,833 

1.45% 

$5,743 

0.21% 

$8,842 

0.40% 

CG-47CL 

$82,848 

3.74% 

$87,424 

1.13% 

$87,424 

1.13%, 

DDG-51CL 

$28,843 

7.08% 

$28,438 

1.15% 

$28,438 

1.15% 

DD-%3CL 

$17,810 

5.14% 

$22,602 

1.20% 

$21,412 

0.38% 

FFG-7CL 

$18,580 

4.08% 

$16,010 

0.61% 

$15,786 

0.56%o 

AFSr50CL 

$1,131 

0.85% 

$1,547 

0.08% 

$1,564 

0.08% 

CVI\S-e8CL 

$81,801 

3.23% 

$23,518 

0.56% 

$23,518 

0.56% 

WEIGHTED  MAPE 

15.14% 

15.39% 

LANTFLEET  SUM 

$187,586 

$166,906 

$167,188 

PACFLEETSUH4 

$143,484 

$168,472 

$167,532 

TOTAL  SUM 

$337,070 

$335,378 

$334,780 

Table  9:  MAPE  Comparison  for  PFAD  and  the  Mod.  Model  SR  2002 
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SR  ■  2001 

Actual  Cost 

Weighting 

PFAD 

PFAD 

Weighted 

MAPE 

Best 

Method's 

Prediction 

Best 

Method's 

Weighted 

MAPE 

Atlantic  Fleet 

AOE-1CL 

13,029 

0.95% 

53,009 

0.01% 

53,009 

0.01% 

AOE-6CL 

13,027 

0.95% 

52,798 

0.08% 

52,991 

0.01% 

MCM-1CL 

56,177 

1.93% 

57,724 

0.39% 

57,724 

0.39% 

MHC-5ia 

56,001 

1.87% 

5449 

23.20% 

51 ,313 

6.70% 

LHA-ia 

53,863 

1.21% 

54,317 

0.13% 

54,479 

0.17% 

LHD-ia 

57,655 

2.39% 

57,147 

0.17% 

57,147 

0.17% 

LPD-4CL 

53,858 

1.21% 

54,191 

0.10% 

54,191 

0.10% 

LSD-36CL 

5876 

0.21% 

5819 

0.04% 

5819 

0.04% 

LSD-41  CL 

53,963 

1.24% 

55,210 

0.30% 

55,210 

0.30% 

CG-47CL 

538,524 

12.04% 

533,635 

1.75% 

533,635 

1.75% 

DDG-51CL 

523,959 

7.49% 

525,786 

0.53% 

521 ,648 

0.80% 

DD-963CL 

525,002 

7.81% 

521 ,155 

1.42% 

523,069 

0.85% 

FFG-7CL 

525,607 

8.00% 

522,486 

1.11% 

523,905 

0.57% 

ARS-50CL 

5395 

0.12% 

5739 

0.06% 

5926 

0.07% 

CVN-68CL 

57,251 

2.27% 

533,458 

1.77% 

526,830 

1.85% 

Pacific  F/eet 

AOE-KL 

$2,359 

0.39% 

$1,655 

0.65% 

$2,691 

0.06% 

AOE-eCL 

$2,317 

0.72% 

$1,339 

0.19% 

$2,372 

0.14% 

LHA-1CL 

$7,350 

2.39% 

$7,129 

0.17% 

$7,577 

0.02% 

LHD-1CL 

$6,279 

1.96% 

$7,122 

0.23% 

$7,122 

0.23% 

LPD-4CL 

$6,006 

1.33% 

$5,653 

0.12% 

$5,653 

0.12% 

LSD-3eCL 

$1,656 

0.52% 

$1,659 

0.00% 

$1,665 

0.00% 

LSD-41  CL 

$5,269 

1.65% 

$5,165 

0.03% 

$3,299 

0.93% 

CG-47CL 

$35,017 

10.94% 

$35,075 

0.02% 

$35,075 

0.02% 

DDG-51CL 

$23,462 

7.33% 

$23,320 

0.04% 

$23,320 

0.04% 

D[P%3CL 

$22,934 

7.13% 

$23,241 

0.03% 

$22,635 

0.11% 

FFG-7CL 

$15,295 

4.73% 

$16,131 

0.26% 

$15,197 

0.03% 

AR^50CL 

$1,520 

0.47% 

$1,593 

0.02% 

$1,449 

0.02% 

CVLmCL 

$30,737 

9.62% 

$31,250 

0.14% 

$31,250 

0.14% 

WEIGHTED  MAPE 

15.28% 

LANTFLEET  SUM 

5158,987 

5172,925 

5166,897 

PACFLEETSUM 

$161,101 

$160,336 

$159,305 

TOTAL  SUM 

5320,088 

5333,811 

5326,702 

Table  10:  MAPE  Comparison  for  PFAD  and  the  Mod,  Model  SR  2001 
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SR  ■  2000 

Actual  Cost 

Weighting 

PFAD 

PFAD 

Weighted 

MAPE 

Best 

Method's 

Prediction 

Best 

Method's 

Weighted 

MAPE 

Atlantic  Fleet 

AOE-1CL 

12,665 

0.83% 

82,340 

0.12% 

82,340 

0.12% 

AOE-6CL 

83,067 

0.98% 

82,447 

0.24% 

83,334 

0.08% 

MCM-ia 

87,509 

2.35% 

87,570 

0.02% 

87,570 

0.02% 

MHC-5ia 

8873 

0.27% 

8480 

0.22% 

8984 

0.03% 

LHA-ia 

84,208 

1.32% 

83,182 

0.42% 

84,297 

0.03% 

LHD-ia 

85,449 

1.70% 

86,125 

0.19% 

86,125 

0.19% 

LPD-4CL 

84,125 

1.29% 

83,378 

0.29% 

83,378 

0.29% 

LSD-36CL 

8793 

0.25% 

8743 

0.02% 

8743 

0.02% 

LSD-41  CL 

84,924 

1.54% 

84,800 

0.04% 

84,800 

0.04% 

CG-47CL 

832,430 

10.14% 

832,459 

0.01% 

832,459 

0.01% 

DDG-51CL 

821 ,917 

6.85% 

820,220 

0.58% 

820,318 

0.54% 

DD-963CL 

824,236 

7.58% 

822,508 

0.58% 

825,457 

0.36% 

FFG-7CL 

822,179 

6.94% 

820,814 

0.45% 

823,216 

0.31% 

ARS-50CL 

8889 

0.28% 

8562 

0.16% 

8828 

0.02% 

CVN-68CL 

830,087 

9.41% 

830,070 

0.01% 

827,812 

0.77% 

Pscific  F/eet 

AOE-1CL 

0.30% 

$1,443 

0.10% 

$2,271 

0.10% 

AOE-6CL 

^2,635 

0.34% 

$1,500 

0.57% 

$2,424 

0.00% 

LHA-1CL 

$3,035 

2.51% 

$3,372 

0.33% 

$7,040 

0.35% 

LHD-^CL 

$e,543 

2.05% 

$7,725 

0.31% 

$7,725 

0.31% 

LP!^4CL 

$5,4% 

1.72% 

$5,417 

0.02% 

$5,417 

0.02% 

LSD-%CL 

$1,704 

0.53% 

$1,443 

0.10% 

$1,534 

0.03% 

LSD-41  CL 

$5,437 

1.70% 

$4,443 

0.33% 

$4,407 

0.40% 

CG-47CL 

$35,723 

11.17%o 

$20,013 

2.53%o 

$20,013 

2.53%o 

DDG-51CL 

$10,532 

3.12% 

$10,250 

0.10% 

$10,250 

0.10% 

D[P%3CL 

$25,323 

3.03% 

$10,202 

2.70% 

$22,330 

1.25% 

FEG-7CL 

$14,512 

4.54% 

$14,011 

0.13% 

$14,300 

0.03% 

AREr50CL 

$1J44 

0.55% 

$1,311 

0.13% 

$1,440 

0.11% 

CVI^mL 

$25,273 

7.00% 

$21,732 

1.20% 

$21,732 

1.20% 

WEIGHTED  HAPE 

12.^7% 

9.50% 

LANTFLEET  SUM 

8165,351 

8157,697 

8163,663 

PACFLEETSUHf 

$154,447 

$132,075 

8139,25(? 

TOTAL  SUM 

8319,798 

8290,672 

8302,913 
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Table  11:  MAPE  Comparison  for  PFAD  and  the  Mod,  Model  SR  2000 


SO  ■  2002 

Actual 

Cost 

Weighting 

PFAD 

PFAD 

Weighted 

MAPE 

Best 

Method's 

Prediction 

Best 

Method's 

Weighted 

MAPE 

Atlantic  Fleet 

A0E-1CL 

12,835 

1.22% 

$3,043 

0.08% 

$3,043 

0.08% 

AOE-6CL 

$466 

0.20% 

$767 

0.08% 

$304 

0.11% 

MHC-5ia 

$848 

0.37% 

$1 ,153 

0.10% 

$570 

0.18% 

LHA-ia 

$7,378 

3.18% 

$4,614 

1.91% 

$5,387 

1.18% 

LHD-ia 

$11,288 

4.87% 

$9,699 

0.80% 

$7,596 

2.37% 

LPD-4CL 

$6,339 

2.74% 

$5,123 

0.65% 

$4,260 

1.34% 

LSD-36CL 

$1 ,525 

0.66% 

$705 

0.77% 

$1 ,323 

0.10% 

LSD-41  CL 

$6,855 

2.96% 

$4,214 

1.85% 

$4,073 

2.02% 

CG-47CL 

$16,497 

7.12% 

$12,418 

2.34% 

$13,202 

1.78% 

DDG-51CL 

$17,606 

7.60% 

$13,319 

2.45% 

$11,458 

4.08% 

DD-963CL 

$9,412 

4.06% 

$8,114 

0.85% 

$10,281 

0.34% 

FFG-7CL 

$12,952 

5.59% 

$9,292 

2.20% 

$10,848 

1.21% 

ARS-50CL 

$1 ,220 

0.53% 

$943 

0.15% 

$1 ,121 

0.05% 

CVN-68CL 

$40,720 

17.57% 

$41 ,281 

0.24% 

$41 ,281 

0.24% 

Pacific  F/eet 

AOE-1CL 

$2,eoi 

1.12% 

$2,313 

0.12% 

^2,913 

0.12% 

AOE-eCL 

$2,44^ 

1.0&% 

$2,337 

0.10% 

$2,742 

0.11% 

LHA-1CL 

$7,%3 

3.23% 

$3,233 

0.31% 

$10,333 

0.34% 

$7J12 

3.07% 

$3,500 

0.50% 

$3,573 

0.73% 

LPD-4CL 

$e,e&7 

2.33% 

$3,313 

0.57% 

$3,373 

0.74% 

LSEP^eCL 

$2,714 

1.17% 

$2,351 

0.13% 

$2,533 

0.05% 

iSD-4^CL 

$4,7&4 

2.07% 

$3,307 

0.33% 

$4,304 

0.05% 

CG-47CL 

$12,10e 

5.22% 

$14,227 

0.73% 

$11,423 

0.31% 

DDG-51CL 

$11,147 

4.31% 

$15,330 

1.33% 

$13,344 

0.33% 

DD-%3CL 

$3,350 

3.30% 

$3,333 

0.50% 

$10,032 

0.32% 

EFG-7CL 

$3,4&5 

3.35% 

$7,333 

0.27% 

$3,133 

0.23% 

ARS-50CL 

$1,231 

0.53% 

$1,370 

0.05% 

$1,537 

0.12% 

CVfmCi 

$20,310 

3.33% 

$22,157 

0.32% 

$22,157 

0.32% 

20.58% 

20.70% 

LANTFLEET  SUM 

$135,941 

$114,683 

$114,546 

PACFLEET  SUM 

$95,809 

$111,981 

$110,424 

TOTAL  SUM 

$231 ,750 

$226,884 

$224,970 
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Table  12:  MAPE  Comparison  for  PFAD  and  the  Mod.  Model  SO  2002 


Table  13:  MAPE  Comparison  for  PFAD  and  the  Mod.  Model  SO  2001 


Table  14:  MAPE  Comparison  for  PFAD  and  the  Mod.  Model  SO  2000 
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D,  RESULTS  AND  CONCLUSION 

From  the  above  data,  Table  15  below  summarizes  the  Weighted  MAPE  for  eaeh 

year. 


SR 

PFAD 

Mod,  Model 

2002 

15.14% 

15.39% 

2001 

33.01% 

15.28% 

2000 

12.67% 

9.50% 

Mean 

20.27% 

13.39% 

SO 

PFAD 

Mod,  Model 

2002 

20.58% 

20.70% 

2001 

20.91% 

23.41% 

2000 

19.34% 

14.97% 

Mean 

20.27% 

19.69% 

Table  15:  Weighted  MAPE  Summary 


The  above  results  demonstrate  that  the  modified  model  is  able  to  lower  the  overall 
MAPE  verses  the  PE  AD  MAPE  for  SR.  Eor  SO,  the  modified  model  is  able  to  lower  the 
MAPE  only  fraetionally.  We  feel  that  these  results  are  appropriate  given  the  foeus  of  our 
study.  Though  we  were  able  to  establish  relationships  between  SR  eost  and  operational 
data  for  several  ship  elasses,  the  optimal  MAPE  was  generally  the  result  of  regressions 
with  EY  as  an  independent  variable.  This  relationship  replaees  the  current  methodology 
of  three-year  average  with  a  regression  equation.  Though  we  did  not  observe  the  im¬ 
provement  we  had  hoped  for  in  the  SO  model,  we  feel  this  is  caused  partially  by  the  na¬ 
ture  of  spending  in  this  Special  Interest  Item.  SR  cost  is  driven  by  specific  material  or 
inventory  deficiency.  SO,  on  the  other  hand  has  a  tendency  to  be  more  discretionary. 

Given  the  above  results,  we  recommend  using  a  regression-based  model  to  predict 
cost  for  SR.  Eurther,  we  also  recommend  implementation  of  a  regression  based  model 
for  SO  prediction.  Though  the  improvement  in  MAPE  is  negligible,  the  increased  flexi¬ 
bility  in  the  modified  model  represents  an  improvement  worthy  of  implementation. 
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